QUATERNION H-TYPE GROUP AND DIFFERENTIAL OPERATOR A 
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Abstract. We study the relations between the quaternion H-type group and the boundary 
of the unit ball on two dimensional quaternionic space. The orthogonal projection of the 
space of square integrable functions defined on quaternion //-type group into its subspace of 
boundary values of g-holomorphic functions is consider. The precise form of Cauchy-Szego 
kernel and the orthogonal projection operator is obtained. The fundamental solution for the 
operator Aa is found. 



Dedicated to Professor Qikeng Lu on his 80th birthday 

1. Introduction 

The real number system is extended to the complex then to the quaternion systems of 
numbers and further finds its the most exciting generalization incorporated the geometric con- 
cept of the direction, the so-called Clifford algebras, which Clifford himself called "geometric 
algebras", |17l I18j . see also |13| for Clifford analysis and numerous references. The division 
algebras, which are only real, complex, quaternionic and octonionic numbers give the origin 
to homogeneous groups satisfying J 2 condition jSJ. The simplest non-commutative example of 
them is the Heisenberg group closely related to the complex number system and that found 
its numerous applications in physics, quantum mechanics, differential geometry (see, for in- 
stance 0E3EI])- The following more complicate example is an quaternion analogue of the 
Heisenberg group that has at least four dimensional horizontal distribution and three dimen- 
sional center We call this group quaternion H-type group due to [Tlj . 

The quaternion H-type group Q can be realized as a boundary of the unit ball on the two 
dimensional quaternionic space HI 2 . The Siegel upper half space U\ £ H 2 is g-holomorphically 
equivalent to the unit ball in H 2 . The group Q arise as the group of translations of U\. This 
leads to its identification with the boundary dlA\. We give the precise formulas of the action. 
Because of this identification and by the use of various symmetries of IA\ the Cauchy-Szego 
projector is realized as a convolution operator on the group Q with explicitly given singular 
kernel. The analogue for the group Q of the Laplace operator is the so called Aa operator 
that is expressed as a sum of the square of vector fields forming the frame of the horizontal 
distribution plus the A times their commutators. We find the kernel of the operator A\. 

Part of this article is based on a lecture presented by the first author during the International 
Conference on Several Complex Variables which was held on June 5-9, 2006 at the Chinese 
Academy of Sciences, Beijing, China. The first author thanks the organizing committee, 
especially Professor Xiangyu Zhou for his invitation. He would also like to thank all the 
colleagues at the Institute of Mathematics, AMSS, Chinese Academy of Sciences for the warm 
hospitality during his visit to China. We also would like to thank Professor Jingzhi Tie for 
many inspired conversations on this project. 
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2. Quaternion i7-TYPE group and the Siegel upper half space. 

We remember shortly the definitions of quaternion numbers. Let 11,12,13 be three imaginary 
units such that 

%\ = %\ = %\ = % x i 2 i z = -1. 
The multiplication between the imaginary units is given in Table ^ Any quaternion q can 

Table 1. Multiplication of imaginary units 
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be written in the algebraic form as q = t + a%\ + bk + ck, where t,a,b,c are real numbers. 
The number t is called the real part and denoted by t = Reg. The vector u = (a, b, c) is the 
imaginary part of q. We use the notations 

a = Imi Q, b = Im.2 q, c = IHI3 q, and Iraq = u = (a, b, c). 

Similarly to complex numbers, vectors, and matrices, the addition of two quaternions is equiv- 
alent to summing up the elements. Set q = t + u, and h = s + xi\ + yk + ZI3 = s + v. 
Then 

q + h = (t + s) + (u + v) = (t + s) + (a + x)k + (b + y)i 2 + (c + 2)« 3 - 
Addition satisfies all the commutation and association rules of real and complex numbers. The 
quaternion multiplication (the Grassmanian product) is defined by 

qh = (ts — u • v) + (tv + su + u x v) , 

where u • v is the scalar product and u x v is the vector product of u and v. The multiplication 
is not commutative because of the non-commutative vector product. The conjugate q to q is 
defined in a similar way as for the complex numbers: q = t — ai\ — bk — 013. Then the modulus 
\q\ of q is given by \q\ 2 = qq = t 2 + a 2 + b 2 + c 2 . The scalar product between q and h is 

(2.1) (q, h) = He(qh) = ts + ax + by + cz. 

We also have 

qh = hq, \qh\ = \q\\h\, q' 1 = An- 

\Q\ 

The imaginary units have the representation by (4 x 4) real matrices: 

-1 " 
0-1 
10 0' 
10 

Then a quaternion can be written in the matrix form 

= t\J + aii + bk + ci 3 , 

where U is the unit (4 x 4) matrix. Notice that 
1. detQ = \q\\ 
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2. Q T = — Q represents the conjugate quaternion q, 

4. Q" 1 represents the inverse quaternion g _1 . 

The nice description of algebraic and geometric properties of quaternion a reader can find in 
the original book of W. Hamilton [7]. 

Let H 2 be a two dimensional vector space of pairs h = (hi, h 2 ) over the field of real numbers 
with the norm \\h\\ 2 = hf + h 2 .. We describe the Siegel upper half space in H 2 carrying out the 
counterpart with the two dimensional complex space C 2 . 

Let D\ denotes the unit ball in C 2 : 

£>i = {(«?!, 102) G C 2 : |^i| 2 + 1 2 < 1}. 

The set 

Ui = {(zi,z 2 ) eC 2 : Rez 2 > |^i | 2 > 

is the Siegel half space. The Caley transformation 

2zi 1 — Z 2 

wi = — , w 2 = — , 

1 + 2:2 1 + 2:2 

and its inverse 

w\ 1 — w 2 

1 + U>2 l + W 2 

show that the unit ball D\ and Siegel half space U\ are biholomorphically equivalent. 
Now, take the unit ball B\ in H 2 

Bi = {(h 1 ,h 2 ) em 2 : |/ii| 2 + |/i 2 | 2 < 1} 

and a Siegel half space in H 2 

^i = {(<7i,9 2 )eIHI 2 : Reg 2 >ki| 2 }- 

The Caley transformation, mapping the unit ball B\ to the Siegel half space U\ and vice versa 
has the form: 

7 (, , n , s-i/-, s\ 2qi(l + q 2 ) , , lf , (l+q 2 )(l-q 2 ) 
hi = qx(l + (1 + q 2 ) (l-q 2 ))=—— — , h 2 = (1 + q 2 ) 1 - q 2 ) = — = , 

|1 + 92| |1 + 92| 

and the inverse transformation 

q 1 = h 1 {l + h 2 ) = _ , g 2 = (1-7*2 + = h , , | 2 • 

|1 + «2| |1 + «2| 

Since the multiplication of quaternion is not commutative, it is possible to define another 
Caley's transformation, but the geometry will be the same. The boundary of IA\ is 

aWi = q 2 ) GH 2 : Re</ 2 = |<Zi| 2 }. 

We mention here three automorphisms of the domain U\, dilation, rotation and translation. 
Let q = (qi, q 2 ) G U%. For each positive number 5 we define a dilation 5 o q by 

5 o ? = 5 o (gi,g 2 ) = (^gi, 5 2 g 2 )- 

The non-isotropy of the dilation comes from the definition of IA\. For each unitary linear 
transformation 1Z on EI we define the rotation TZ(q) on IA\ by 

K{q)=K{q x ,q 2 ) = (K(qi),q 2 ). 

Both, the dilation and rotation give q-holomorphic (the definition of g-holomorphic mapping 
see below) self mappings of U\ and extend to mappings on the boundary dU\. Before we 
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describe a translation on U\, we introduce the quaternionic H-type group denoted by Q. This 
group consists of the set 

Ixl 3 = {[w,t] : weM,t= {h,t 2 ,h) € 

with the multiplication law 

(2.2) [w,ti,t2,t 3 ]-[uj,si,S2,S3} = [w+uJ,ti+si-2ImiOw,t2+S2-21m 2 u!W,t 3 + ss-2Im 3 u!w]. 

The law (I2.2j) makes Ixl 3 into Lie group with the neutral element [0, 0] and the inverse 
element given by [w, t\, £3] = [— w, — 1\, — 1%, — t^]. 

To each element [w, t] of Q we associate the following g-holomorphic affine self mapping 
ofWi. 

(2.3) [w, h,t2,t 3 ] : (qi,q 2 ) i-> (q 1 +w,q 2 + \w\ 2 + 2wq 1 + i x t\ + z 2 *2 + hh). 
This mapping preserve the following "height" function 

(2.4) r(q) =Req 2 - \qi\ 2 . 
In fact, since \qi + w\ 2 = \qi\ 2 + \w\ 2 + 2Reu)(/i, we obtain 

Re((/2 + |^| 2 + 2wq\) - \qi + u>| 2 = Re<? 2 - Iq'i | 2 - 

Hence, the transformation (|2.3j) maps £4 to itself and preserves the boundary dU\. 

The reader can check that the mapping (|2.3|) defines an action of the group Q on the space IA\ . 
If one composes the mappings (|2.3j) . corresponding to elements [w,t], [u>,s] G Q, the resulting 
transformation will correspond to the element [w,t] ■ [u, s]. Thus, (|2.3|) gives us a realization 
of Q as a group of affine g-holomorphic bijections of U\. We can identify the elements of U\ 
with the boundary via its action on the origin 

h(0) = [w, t] : (0, 0) 1 ^ (w, \w\ 2 + hh + i 2 t 2 + i 3 t 3 ), 
where h = [w,t]. Thus 

Q 3 [w,ti,t 2 ,t 3 ] i-> (w, \w\ 2 + iiti + i 2 t 2 + i 3 t 3 ) G dli\. 
We may use the following coordinates (qi,t,r) = {qi,ti,t 2 ,t 3 ,r) on U\\ 

U\ 3 (^1,92) = (9l,*l,*2,<3,»")> 

where 

t\=\w\\q2, £ 2 =Im 2 g 2 , t3 = Im3Q , 2, r = r(q x ,q 2 ) = Req 2 - \qi\ 2 . 
If Reg 2 = |9i| 2 we get the coordinates on the boundary dU\ of the Siegel half space 

dili 3 (91,92) = {qi,ti,t 2 ,t 3 ), 

where t m are as above and r = r(qi, q 2 ) = 0. 

3. Tangential Cauchy-Riemann-Fueter operators 

Before going further, we collect here the necessary definitions concerning the quaternion 
calculus. Q-holomorphic functions on EI were studied by Fueter and his collaborators I12j. 
The reader can find the account of the theory of g-holomorphic functions in 1131 12()j . 

We continue exploit the analogy with the complex variables theory. Let M be a manifold of 
dimension 2n with the complex structure /. Such kind of manifolds is called complex manifold. 
Let / : M — > C be a differentiable function defined on the complex manifold M. Write 
/ = /o + ifi, where fo, fi '■ M — > R. Then / is called holomorphic, if 

(3.1) df + I{dfi) = 
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on M, where ()3.1j) is called the Cauchy-Riemann equation. Let, now, M be a manifold of 
dimention An. A hypercomplex structure on M is a triple (Ji, 12,13) on M, where is a 
complex structure on M and ii/2 = ^3- If Af has hypercomplex structure, it is called the 
hypercomplex manifold. Let / : M — > EI be a smooth function defined on a hypercomplex 
manifold M. Then / = /o + ii/i + ^2/2 + ^3/3, where /o, . . . , f 3 are smooth functions. We 
define a g-holomorphic function on M to be a smooth function / : M — > H for which 

(3.2) df = 4f + Ji(d/i) + l 2 (d/ 2 ) + / 3 (4f 3 ) = 0. 

Equation (|3.2j) is the natural quaternionic analogue of the Cauchy-Riemann equation ()3.1j) and 
is called the Cauchy-Riemann-Fueter equation. 

The equation (|3,2j) can be written in terms of the partial derivatives with respect to the 
quaternionic variable. We introduce the differential operators 



(3.3) 



^L-^f-\(^L^sr- 1L\ d r / _ a t _ 1 fdf , ^ df 

dq ~ U ~ 2\dx Q + ^ %m dxJ' dq r/ ~ 2 \dx + ^ dx ' 

%L_ af _lfdl_x-. df_\ d r f _ _ 1 fdf A &l_ , 

" °" " 2 Ux Z^dxJ' dq ~ ° rI ~ 2 Us ^ 9x m ' r 

i2 



m I 1 



A _ y/ | df 2 | d 2 / | d 2 f 
dxl dx\ dx\ dx\ ' 

Note that di,di,d r and d r all commute, and A = 4d r d r = 4<9;<9/. Since the theories generated 
by di and d r are symmetric, we shall use the left operator d[ and call the function simply 
g-holomorphic or regular. If / : M — ► H a real-differentiable function, the Cauchy-Riemann- 
Fueter equation ()3.2j) can be written as dif = i. e. 

dif = ^ = -Jr-\ +y2 im x? d \ =0 ' fc = 1, 
o(x )fc ^ a(x m ) fc 

We say that vector fields are tangential Cauchy-Riemann-Fueter operators if the following 
holds. 

(i) Vector fields on dU 1 arise by restricting (to dU 1 ) of vector fields that, in coordinates 
(lli 92) of H 2 , can be written in the form 

(3.4) ai—^ + a 2 -^—, 

dqi dq 2 

where the ay are quaternion valued functions. This is the class of first-order differential 
operators that annihilate q-holomorphic functions. 

(ii) Vector fields are tangential at dU 1 in the sense that 

dir(q) dir(q) 

oii—^z V OL 2 _ = 0, 

dqi dq 2 

wherever r(q) = 0. Here r(q) is the defining function given by (|2.4|) . 
We consider the vector fields on the boundary dli\ of the Siegel half space with the coordi- 
nates (q,t) = (xo,x 1 ,x 2 ,x 3 ,ti,t 2 ,t 3 ): 

X (q,t) =d XQ - 2xid tl - 2x 2 d t2 - 2x 3 d t3 , 
Xi (q, t) =d Xl + 2x d tl - 2x 3 d t2 + 2x 2 d t . A , 
X 2 (q,t) =d X2 + 2x 3 d tl + 2x d t2 - 2xid tz , 
X 3 (q,t) =d xs - 2x 2 d tl + 2xid t2 + 2x d t3 . 
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They form a basis of the subbundle T(dU\) of the tangent bundle T(dU\). In the same time the 
vector fields can be considered as a basis of Lie algebra, which is an infinitesimal representation 
of the group Q. The commutators between the vector fields ()3.5j) is given in Table |^1 Let us 



Table 2. Commutators of the vector fields (|3.5|) 
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verify, that the vector fields (|3.5|) are tangential Cauchy-Riemann-Fueter operators. To show 
this, we define the quaternion vector fields 

- 1 .9/ 

H(w, t) = - (X + iiXi + i 2 X 2 + i 3 X 3 ) = — + wiid tl + wi 2 d t2 + wi 3 d ta , 
(3.6) I ™ 

H(w,t) = -(X - i x Xi - i 2 X 2 - i 3 X 3 ) = — - i\wd tl - i 2 wd t2 - i 3 wd t3 , 

where the terms wi k and i k w, k = 1, 2, 3, are the quaternion product. Then the commutative 
relation is [H, H] = 2Y^ k= \ikdt k - 

Lemma 3.1. Using the identification of Q with dU\, the vector field H is tangential Cauchy- 
Riemann-Fueter operator on dli\ . 

Proof. We remember the quaternion coordinates (w,t,r) on U\\ for (qi,q 2 ) € U\ 

w = qi, t k = lva k q 2 , k = 1,2,3, r(q 1 ,q 2 ) = Req 2 - |<ji| 2 . 
We use the left multiplication chain rule 

di diw di diw_di_ d^_ d_ drr_d_ m=1 2 

fin fiin < Fin .. f)f fin ., fir 1 



dq m dq m dw dq m dw ^ dq m dt dq m dr 

Calculating the derivatives 

diw _ diw_ _ dftk _ Q diw_ _ 1 ditk_ _ h dtr_ _ _ drr _ 1 
dq m dq 2 dqi ' dqi dq 2 2' dqi 91 ' dq 2 2' 

for k = 1, 2, 3, m = 1, 2, we conclude 

ch_ _ di_ _ (h_ di _J^i k d[ | 1 di 
dqi dw 91 dr ' dq 2 ^ 2 dt k 2 dr 

Multiplying the second equation by 2qi from the left and summing the derivative, we get 

Q^+l^^md tm = w + 2 qi — . 

m=l 

Notice that ^ + 2qi§j^ = 0. This proves that the operator + 2qi-j^ is tangential Cauchy- 
Riemann-Fueter operator. □ 
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4. The Cauchy-Szego kernel 

We will need the following 3-form Dq, which is different from the usual volume 3- form in M 3 . 
We write v = dxo A dx\ A dx 2 A dx% for the volume form in H. The form Dq is defined as an 
alternating M trilinear function by 

(4.1) (hi,Dq(h 2 ,h 3 ,h4)} = v(h 1 ,h 2 ,h 3 ,h i ) 

for all h\, . . . , /t4 G H. The coordinate expression for Dq is 

= dxi A A dx 3 — i\dxQ A A dx 3 — i 2 dxo A dx 3 A cfai — i3<ixo A dx\ A dx2- 

Other properties of Dq reader can fined in [2131 • This is the principal form for the formulations 
of the integral theorems related to the q-holomorphic functions. We present the Cauchy integral 
theorem here. 

Theorem 4.1 ( 20 ). Suppose f is q-holomorphic function in an open set U E H. Let qo be a 
point in U, and let C be a rectifiable 3-chain which is homologous, in the singular homology of 
U \ {qo}, to a differ entiable 3-chain whose image is dB for some ball B C U . Then 

1 f iq - q ° rl -Dqf(q) = nf(q ), 



271-2 Jc l<?-<7o| 2 
where n is the wrapping number of C about q$. 

We start from the definition of the Hardy space H 2 (Ui). In the sequel, we write i ■ t = 
Y^n=i irntm- Let dh be Haar measure on Q. Using the identification of dtii with Q we 
introduce the measure df3 on dlAi. So, the integration formula 

/ F(q)dP(q)= [ F{qi,\ gi \ 2 + i-t)dqidt, 
JdU! JHxR 3 

holds for a continuous function F of compact support. With this measure we can define the 
space L 2 (Q)= L 2 (dUi). 

For any function F defined on Ui, we write F e for its "vertical translate" (we mean that the 
vertical direction is given by the positive direction of Re^): 

F £ (q) = F(q + ee), where e = (0, 0, 0, 0, 1, 0, 0, 0). 

If e > 0, then F e is defined in the neighborhood of dU\. In particular, F e is defined on dlA\. 

Definition 4.1. The space TC 2 (Ui) consists of all functions F holomorphic on Ui, for which 

(4.2) sup/ \F £ (q)\ 2 dp(q) < oo. 

e>0 JdU! 

The norm H-^H?^^) of F is then the square root of the left-hand side of 1)4.2(1 . The space 
H 2 (Ui) is a Hilbert space. 

We shall need the following statements concerning the holomorphic Ti 2 space on the upper 
half space Ui of M 2 . We denote the upper half space Ui by R%. Define H 2 (R%) to be a set of 
functions f(q) which are q- holomorphic for q = u + i-v in the upper half space u > satisfying 

(4.3) (sup / \f(u + i-v)\ 2 dvY /2 = \\f\\ H 2 (m) < oo. 



The maximal inequality 



(4.4) / sup \f(u + i ■ v)\ 2 dv < A 

JR3 n >Q 
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is a consequence of (j4.3|) . This implies that there exists an f b G L 2 (M 3 ) such that 

(4.5) f(u + i • v) — * f b (v) as u — > 0, for a. e. i>, 
and the identity 

(4-6) II/IlW = ||/|| w2(R 4 +) . 

The definition of W 2 (R^) and further properties can be found in |19| . 

Let ft £ I and <5 > 0. We define / on R\ by the following f(q 2 ) = F( qi ,q 2 + S + |gi| 2 ). 

Lemma 4.2. Suppose that F G TL 2 (U\) and 5 > 0. T/ien /or eac/i oi G EI, £/ie function 

f(Q2) = F( qi ,q 2 + 5+\ qi \ 2 ) 

with q 2 = u + i • v, belongs to 7i 2 (R\) 

Proof. We use the action (|2.3[) of group Q on £/i and others g-holomorphic automorphisms 
of IA\ to simplify the statement. Let h = [w,t] & Q and q = (qi,q 2 ). It is easy to see that 
% + ee) = h(q) + ee. Then F(h(q)) G H 2 (^i) whenever F(g) G H 2 {Ui), since the measure 
d(3{q) is invariant with respect to action of /i. We take /i = [q\, 0] and reduce the considerations 
to the function f(q 2 ) = F(0, q 2 + 5). Applying the dilations and the property d/3(5(q)) = 5 10 df3, 
we can consider 5=1 and work with 

f(u + i-v) = F(0,u + l + i-v), for u > 0. 

By the standard way, from the Cauchy integral Theorem 14. II one can obtain the mean value 
property for g-holomorphic functions 

(4.7) \f(u + i-v)\ 2 = \F(0,u+l+i-v)\ 2 < c J \F(q u q 2 + u + 1 + i • v)\ 2 dq 1 dq 2 . 

ki| 2 +k 2 | 2 <i/4 

where c _1 = is the volume of the ball of the radius 1/2 in H 2 . Notice that since | Req^l < 
1/2 and u > 0, we have Re(q 2 + u + 1) G (1/2,3/2). Thus Re(g 2 + u + 1) > \qi\ 2 and this 
guarantees that the integration in (|4.7|) is taken over some subset of U\ . We write q 2 = x + i • y 
and integrate (|4.7|) with respect to v over R 3 . After applying the Fubini theorem, we deduce 

j Jf(u + i-v)\ 2 dv <c J J J \F(qi,q 2 +u+l + i-v)\ 2 Dq 2 dxd qi dv. 

|qi|<l/2R3 |g 2 |<l/2 

We take now the second integral with respect to Dq 2 and write v = y. We get 

J Jf(u + i-v)\ 2 dv < ci J J J \F(qi,x + u+l + i-y)\ 2 dxdq 1 dy. 

|9i|<l/2R 3 |as|<l/2 

We make the following change of variables x + u + 1 = e+l^il 2 . Since \x\ < 1/2 and j^il 2 < 1/4, 
the rang of new variable e is in the interval (u + 1/4, u + 3/2). So, the last integral take the 
form 

u+1/4 

J j j \F(q u e+\ qi \ 2 + i-y)\ 2 d qi dyde 

u+3/2 |<?i|<l/2 M 3 

<^ J \F( qi ,e + \ qi \ 2 + i-y)\ 2 d qi dy = ^ J \F(q + ee)\ 2 d[3(q) < oo 
IxR 3 8Ui 
because of (|4.2|1 . This shows that / G 7Y 2 (B^_) and we finish the proof of Lemma 14.21 □ 
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Theorem 4.2. Suppose F belongs to TL 2 {Ui). Then 

1. There exists an F b G l?(dlA\) so that F(z + £b)\qu 1 — > -F fe m i/ie L?(dlA\) norm, as 
e->0. 

2. T/ie above mentioned space of F b is a closed subspace of ' l?{dlA\). Moreover 
3 - ll^ b ||L 2 (9Wi) = ||-F||« 2 (Wi)- 

Proof. To proof Theorem 14.21 we exploit the reducing to the one dimensional case. Let us 
fix qi and consider the function F(q) = / (51,92) as a function of 92- We apply the maximal 
inequality ()4.4|) and equality (|4.6|) to the function /(fe) = F(qi,Q2 + <5 + | 2 )- We continue 
to write 92 = £ + i • V- Then 



/ sup a; + 5 + |gi[ 2 + i • y)| 2 < A 2 f \F(q u 5 + \ qi \ 2 + i • y)\< 

JR3 x >0 JR3 



dy. 



We integrate over q\ G H, write x = e, use the definition of d(3 = dq\ dy, and |qi| 2 = Re 52 on 
the dlA\. Then the last inequality can be written as 

f sup \F(q + e + 5)\ 2 d(3(q) < A 2 f \F{q + 5)\ 2 d(5{q). 
JdUi £>0 JdUi 

Letting 5 to 0, we see 



f S up\F(q + e)\ 2 d(3(q)<A 2 \\F\\ 2 n2{Ui) . 
JdUi e>0 y ' 



IdUi 

l'2\ 



We conclude from the last inequality that for almost all q\ G H the function F(q\,q2 + \qi\ 
as a function of q2, is in ri 2 (W^_) and by Q4.5|) we see that the limit lim e ^o F(q + ee) = F b (q) 
exists for almost every q G dU\. 

We show now the property 3. By Fatou's lemma, it follows 

/ \F b (q)\ 2 dP(q) <sup / \F(q + ee) | 2 df3{q) = \\F\\ 2 H2{Ui) . 
JdUi £>0 JdUi 

From the other hand, we use 1)4.6 J) and get 

/ \F( qi ,e+\ qi \ 2 + i-y))\ 2 dy < sup / |F(gi,e+|gi| 2 + i-y))| 2 = / \F( qi ,\ gi \ 2 + i-y))\ 2 dy. 

JR3 £>0 JR3 JR3 

Integrating over gi € H both sides of the last inequality and taking supremum over e > 0, we 
get the property 3. 

We showed that the limit F b in U 2 {U\) norm exists. To complete the proof of the assertion 2 
we note that the mean value property ()4.7j) says that for any compact K $LlA\ there is a constant 
ck, such that 

(4-8) sup|F(g)[ < c K \\F\\ H 2 {Ul) . 

We conclude, that if a sequence F n converges in H 2 (Ui) norm, then the sequence F n converges 
uniformly on compact subsets of U\, that implies that the space TC 2 (Ui) is complete with 
respect of its norm. The Theorem 14.21 is proved. □ 

We determine now the Cauchy-Szego kernel S(q,u) for the domain U\. The Cauchy-Szego 
kernel S(q,Lo) is a quaternion valued function, defined on U\ x U\ and satisfying the following 
conditions 

1. For each u> G U%, the function q 1— > S(q,u) is g-holomorphic for q G Ui, and belongs 
to H 2 (Ui). This allows us to define, for each uj G IA\, the boundary value S b (q,u>) for 
almost all q G dU\. 
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2. The kernel S is symmetric: S(q,u) = S(lo, q) for each (g, to) G U\ x U\. The symmetry 
permit us to extend the definition of S(u,q) so that for each q G U\, the function 
S b (q,io) is defined for almost every uj G dlA\. 

3. The kernel <S satisfies the reproducing property in the following sense 

(4.9) F(q)= f S{q,Lj)F b (uj)d(3(uj), qeU h 

JdUx 

whenever F G H 2 {Ui). 
Theorem 4.3. Let S(q,uj) = 5( k , , where 



Q2 + w 2 3 
qh ana A; = — -„ 

T/ien S(q,Lo) is the (unique) function that satisfies the properties 1. — 3. above. 



(4.10) r(q,u) = - wigi and k 



Proof. We introduce the Cauchy-Szego projection operator C. The operator C is the orthog- 
onal projection from L 2 (dlA\) to the subspace of functions {F b } that are boundary values of 
functions F G H 2 {Ui). So for each / G L 2 (dU 1 ), we have that C(/) = F b for some F G ft 2 (^i); 
moreover, C(F b ) = F b and C is self-adjoint: C* = C. 

We fix q G Wi< Then (Cf)(q) = F(q), where F corresponds to F . The kernel S(q,ui) will 
be defined by the representation 



(4.11) F(q)= / S(q,u)f(u)dP(u). 

JdUi 

The space TL 2 (Ui) is identified with some subspace of L 2 (dU\). We take a basis {(fj} of this 
subspace. We can expect that 

(4.12) S(q,u)=Y t <p j (q)<p j (u,). 

j 

(In this case we also have the symmetry property 

S(q,u) =^(pj{q)(f>j{u) = ^ip j {u)Cp j {q) = S{u,q). 
j j 

Indeed, if J2j \ a j\ 2 < °°> then J2j a j¥j £ ^ 2 (^i) an d II J2j a j i Pj\\n 2 {U l ) = J2j \ a j\ 2 - Moreover, 
for any compact set K C IA\ we have by (|4.8j) that 

sup I ^2a,j(pj(q) <c A '(^|aj| 2 ) ' ■ 
q£K j j 
Applying the converse of Schwart's inequality, we get 

1 /2 

\fj(q)\ 2 ^j < ck, for all q G K. 

j 

Thus the sum (|4.12() converges uniformly whenever (q,Lo) belongs to a compact subset of 
UixUi. 

Let us take ()4.12() as the definition of S(q,co). By the symmetry S(q,Lu) = S(uj,q) the 
function S(q,uj) belongs to TL 2 iUi) for each fixed qdlA\. Moreover S(q,u>) extends to q G U\, 
uj G dU\, by the identity (|4.12|) with the series converging in the norm of L?{dlA\). 

These arguments establish the existence of the function S satisfying (|4.11|) and the proper- 
ties 1.-3. The reproducing property (|4.9|) uniquely determines S(q,u>) as an element of H 2 {Ui) 
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for each fixed q. Together with conclusion 3 of Theorem 14,21 this shows that S is uniquely 
determined by the properties 1.-3. 

We continue the proof establishing the precise shape of the function S(q,uj). We use the 
translation h(q), the unitary rotation 1Z(q) and the dilation 5(q) on U\. Observe that the 
measure d(3 is invariant with respect to translation and unitary rotations and df3(S(q)) = 5 10 df3, 
where 10 is the homogeneous dimension of the group Q. Notice that the space H 2 (Ui) is also 
takes into itself under the above mentioned transformations. Then, we have 

F(q)= [ S(h(q),h(uj))F h (u;)df3(uj), 
F(q) = [ S(K(q),K(uj))F b (uj) d(3(u), 



F(q)= / S(5(q),S(uj))5^F b (uj)d(3(uj). 

JdUi 

We conclude 

(4.13) S(q,uj) = S(h(q),h(uj)), S(q,u) = S(K(q),K(uj)), S(q,uj) = S(5(q),5(uj))5 w . 

The identities (|4.13j) hold for each (q,uj) € U\ X 1A\. They also hold for each q G U\ and 
almost all uj £ dUi, because for each fixed q£lA\ the identities (|4.13|) are the identities for the 
elements of L 2 (dU\) and, therefore, they are fulfilled for almost all uj E dU\. 

Let S(q) = S(q,0), then S(q) is holomorphic on U\ and is independent of q\. So, we may 
write S(q) = s{q2)- Using the transformation of S with respect to dilation we get s{5 2 q2) = 
5~ 10 s(q2) for all positive 5. We conclude that S(q) = cq^ 5 - We use now the translation h(q). 
Let q £ U\ and uj G dUi. Then by identification of U\ with Q we have uj = h(0) and 
S(q,h(0)) = S(h~" 1 (q), 0) = S(h~ 1 (q)). We calculate the element h such that h(0) = {ui,u)2)- 
We have 

h = [w,t] = [cji,lmi(u;2 - |cji| 2 ), Im 2 (cj2 - |^i| 2 ), Im 3 (u;2 - |^i| 2 )]. 
We also observe, that Reu;2 = \uJi\ 2 - Then the second coordinate of the action h^ 1 is 

q 2 + \uji\ 2 - i • Imw 2 - 2udiqi = q 2 + uj 2 - 1uj\q\ = 2r(q,uj) 

by ((231). Therefore S(q,uj) = kr~ 5 (q,uj) with k = 2~ 5 c. 

We calculate the constant k. Use the reproducing formula for the function F(q) = [q 2 + 
l) -5 = (2r(e, (?)) -5 . Applying the reproducing formula, we get 

2~ 5 = F(e) = k [ r(e,q)- 5 F(q)df3q 
JdUi 

= 2 5 k [ \F(q)\ 2 d(3q = 2 5 k [ [ \F(q' ,\q'\ 2 + V i m t m )\ 2 dq'dt. 

JdU! Jq'eW Jtm 3 m=1 



Thus 

(4.14) k- 1 = ^f dw [ (|t| 2 + (|u;| 2 + l) 2 )~ 5 ^. 



Taking the integral over M 3 , we observe that (j4.14l) is reduced to the product of four values 
a, (3, 7, 5, where a = Air is the volume of the unite sphere in M 3 , 

(3 = 2ir 2 is the volume of the unite sphere in H, 

f°° r 2 dr r(3/2)r(7/2) 

7 = 7o FTIF = 2r (5 ) ' and 
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p 3 dp r(5) 



Multiplying all values, we get k = gfr. □ 
Notice the following. 

(i) The function r(q,u>) is g-holomorphic in q and anti q-holomorphic in u. If q = u, the 
function r agrees with the function (|2.4[) , 

(ii) For each fixed w 6W] the function r(q,co) (and hence ^(g,^)) is g-holomorphic for q in 
a neighborhood of closure ofU\. For each fixed q EU\ the function r(q,u>) (and hence 
S(q,uj)) is anti g-holomorphic for w in a neighborhood of closure of 1A\. In particular, 
if g S Wi is fixed, the boundary function S b (-,u>) is, actually, defined on all of the 
boundary dU\. 

5. The projection operator 

We describe the projection operator C as a convolution operator on the group Q. We 
remember that mapping / i— > C(/) assigns to each element / G L 2 (dlA\) another element 
of l?(dU\) of the form C(/) = for some i 7 E 7i 2 (Ui). Theorem 14.21 and reproducing 
property (|4.11|) imply 

(5.1) (Cf)(q) = limF(q + e)\ dUl = lim / S(g + e, u)f{u) d(3(u), 

where q E <9Wi and the limit is taken in L 2 (dlA{) norm. 

We now use the identification of dU\ with Q. We write dlA\ 3 to = g(0) for the unique 
g E Q, dU\ B q = h(0) for h £ Q, and d(3(u) = dg. The properties Q4.13JI of the Cauchy-Szego 
kernel give 

S(q + e,<j) = S(g-\q + e), g~ x {<*>)) = S{g-\q) + e,g-\u>)) = ^GT^O)) + e, 0). 

We change the notation setting K e (/i) = SO -1 0(0)) +e,0), f(g) = f{g(0)) = f(u), and 
(Cf)(h) = (Cf)(h(0)) = (Cf)(q). Then {□) takes the form 

(5.2) (C/)(/ i ) = lim / K e {g- X o h)/( 5 ) d/%), 

for / E L 2 (Q), where the limit is taken in L 2 (Q). We see that ()5.2|) is formally written as the 
convolution (C f){h) = {K * f)(h), where K = \\m £ ^QK e is a distribution. 

We obtain the precise form of K(h). Let h = [w,t], then by identification of dlA\ with Q 
and Theorem 14.31 we have K £ (h) = c(\w\ 2 + e + i • t)~ 5 with c = We observe that 

Ke(h) = - s f - ((ki 2 + £ + i • t)- 2N 



The function (ju)j 2 +e + i-i)~ 2 is locally integrable on Q (see remark below). Passing to the 
limit, we see that the distribution K is given by 

K(h) = — - f - f(| W | 2 + i-t)- 2 
v ; 24 dt 3 dt 2 dh V u 1 y 

and equals to the function (|il>| 2 + i • i)~ 5 away from the origin. 
Remark 5.1. On the group Q we can introduce the homogeneous norm 

INI 2 = ll[w,*]|l a = H a + l<l. 
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where | • | denotes the Euclidean norm. The homogeneous norm is a homogeneous of order 1 
with respect to dilation 5 function, namely: ||<5(^)|| = S\\h\\. We also recall the analogue of 
the integration formula in the polar coordinates ^Uj. There is a positive constant k, such that 
whenever / is a non-negative function on (0, oo), then 

/poo 
f(\\h\\)dh = k J f(r)r®- l dr, 

Q 

where Q is the homogeneous dimension of the group Q. The homogeneous dimension of Q 
equals 10. Since the kernel of the Cauchy-Szegd projection satisfies |if ~ ||ft<||~ 10 then the 
function (\w\ 2 + i • t)~ 2 is locally integrable. 

Remark 5.2. We can change the arguments that we used for the calculation of the precise form 
of the distribution K(h). It is sufficient to differentiate only one time to obtain the integrable 
function. We can consider 

K £ (h) = -^—((\w\ 2 + e + i-t)- 4 ) for any m = 1,2,3 
and then argue as above. 



6. The kernel of the operator A a 
Recall from Section 3, the operator 

- 1 d 

H ( w i t) = 2 ( x o + hXi + 12X2 + 13X3) = — + wiid tl + wi 2 d t2 + wi3d t . i , 

where the terms wik, k = 1,2,3, are the quaternion product. The conjugate operator is 

1 d 
H(w,t) = -(X Q - i x Xi - i 2 X 2 - 13X3) = — - hwd tl - i 2 wd H - i3wd ts . 

Then the commutative relation give us [H, H] = —2 Ylk=i ^kdt k - 
To define db operator on Q we take a function / and set 

B b f = Hfdq, 

where dq = dxo — Y2k=i ^k^Xk- It is easy to see that 

dq Adq = i\dx 2 A dx3 + i 2 dx3 A dx\ + i3dx\ A dx 2 . 

Then 

dlf = HHfdqAdq^O. 

This is the difference with the Heisenberg group. We then define the formal adjoint 8^ of db 
by 

&lf = —Hf where / = / dq is a (0, l)-form. 
Finally, we can define operator by 

a b = d b d* b +d* b d b . 

When / is a function, then 

B* b B b (f) = —HH(f), 

d b 8* b (f) = o. 
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It follows that 



-HH = ~(HH + HH) + ±[H, H] = -\{HH + HH) - 2 £ z fe «\ 

" fc=l 



i=0 fe=l 



where Xi, I = 0,1,2,3 are defined in (|3.5|) and dt k , k = 1, 2, 3 are their commutators. In fact, 
we may define a little bit more general operator A\ as follows 

3 3 

A A = ^X 2 + 4^ fc A fe d tfe . 

1=0 k=l 

The operator A\ is called the subLaplacian in the literature and from a theorem of Hormander jSj, 
it is a subelliptic operator. In terms of coordinates on Q, one has 

3 3 3 

a a = + 4 N 2 E<t + 4 E ■ ^) + A ^R- 

Z=0 fc=l fe=l 

Let us observe that A^ possesses the following symmetry properties 

(i) is left invariant on Q with respect to the translation defined by the group multiplication 
from the left, 

(ii) has degree 2 with respect to the dilation 5, 

(iii) is invariant under the unitary rotation K on M 4 . 

Recall that partial Fourier transform in the variables t/., k = 1,2,3, is defined as follows 

F(f)(x,T 1 ,T 2 ,T 3 ) = f(x,Tl,T 2 ,T 3 )= [ e~ ^ l » T ^ f (t) dt, 

f(x,h,t 2 ,t 3 ) = / e^ ikt ^f(r)dT. 

(2ttY J r3 

Then 

•^(J^)0£,n,"7- 2 ,T 3 ) =i k r k J{r), f(^J^(x,t 1 ,t 2 ,t 3 ) = -r|/(r). 
Consequently, 

4 3 3 

A a( t ) = E d X ~ 4 ' x l 2 E r fc + 4 E ^ xik ' d ^ ik ~ Afc ) Tfc - 

1=1 k=l k=l 

We write |r| 2 = X^l=i r f • When A = 0, one has 

4 3 

Ao(r) =E<% -4|x| 2 |r| 2 + 4^(i fc (xi fc -^))a tfc . 

Z=l A:=l 

The property (iii) implies that A^(t) is invariant under the unitary rotation group action on 
R 4 . Therefore, the fundamental solution K\(x,t) of A^(t) is a radial distribution. It follows 
that 

3 

E ( xik ' d x )i k T k K x {x) = 0. 
fc=i 
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Hence the operator A>(r) can be reduced to a Hermite operator in R 4 : 

4 3 3 

(6.1) H x (r) = £ &l - 4|*| 2 £ r 2 k - 4 £ A fc r fc . 

Z=l fc=l fc=l 

Now by a result in [Hj, we know that the fundamental solution K\(x) of the operator H x (t) 
has the following form 

Irl 2 Z" 00 „/s-3 x x e -MM 2 cath(4|T|a) 
K x (x,r) = ilL / ds 
vr 2 7 smh 2 (4|r|s) 



where |r| 2 = X^fc=i r fe ■ ^ follows that 

K x {x,t)=-^ [ e^ iktkTk K x (x,r)dT 



I /• /■oo I 1 2 

11 ' |T-p| 2 coth(4|r|s) e ELi(»fc*fc- 4A fc s )' r &d S( i T . 



(6.2) 

8vr 5 7 R 3 7 sinh 2 (4|r|s) 
Let us look at these integrals more carefully. Changing variable 4\t\s = u implies that 

Irl f°° fx^s , v u e -MM 2 coth(«) 



Then 



K x (x,t)=-±- [ e^ lktkTk K x (x,T)dT 
(2vr) d J R3 



s - : Z" 00 r v^3 x ^ i « e -|r||a:| 2 coth(u) 

e E*=i ****** / e -(L fe =i^) H « | T | dudr 



(2vr) 5 7 R 3 7 sinh 2 (u) 

We introduce the polar coordinates for the r-variable such that r = \r\n = \r\(ni,n2, W3) = rn 
where r = \t\. Then the above integral can be rewritten as 



Kx (x,t)=—^ / e r (£*=i / e -(5:*=xA*n fc )u-r[x[ a coth(«) ' du da dr 

( 27 5 Jo Js 2 Jo sinh^(w) 



(27r) 5 7 sinh 2 (ti) 
6 /■ /■» e-EL^^)" duda 



(H 2 coth(«)-ELi i***n»)r r 3 dr du d(J 



(2vr) 5 y 5 2 J sinh 2 (u) [|x| 2 coth(u) - ^Li ifc**"*] 4 ' 

Here d<7 is the surface measure on S" 2 . When A fc = for k = 1,2,3, the integral can be 
simplified. Let v = coth(u), then dv = — sin ^"^ M ^ an d the above inner integral reduces to 

dv 11 



[\x\ 2 v - Y%=\ iktknk} 4 3\x\ 2 [\ x \ 2 - J2l=i iktWkY 
Hence 

da 



If we put i\ = i, the usual complex unity and suppose that 12, h are absent, then the group 
Q is reduced to the 2-dimensional Heisenberg group C 2 x R with a 4-dimensional horizontal 
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space and an one dimensional center. In this case the formula Q6.2|) reduced to the known 
formula obtained in 10 j. Indeed, the Hermit operator 1)6.1 Jl is written in the form 

4 

#a(t)= J> 2 ; -4|x|V-4Ar. 
i=i 

Then 



K x {x) =-?- [ e itr K x (x)dT 

to JR 



= _L / e itr / e ~4Xrs M e -\r\\x\* coth(4|r| S ) dg dr 

2vr 3 A 7o sinh 2 (4|r|s) 
Changing variable 4s to s, the above formula becomes 

8vr d 7 R 7 smh z (|r|s) 

= _L / e ^ T 2 / e-rW dsdr, 

8tt J r J sinh^(rs) 

since |r| 2 , sinh 2 (|r|s), |r| coth(|r|s) are all even functions. Let u = ts, then one has 

8vr J 7 M s 2 7 smh>) s 

/ e -A« / s _3 e -lHz| 2 coth(«)- iiu ) ds ^ 



8vr 3 sinh 2 (u) 

Once again, changing variable - to s in the second integral, then we have 

«OC POO 

s ~3 e --(u\x\ 2 coth(u)-itu) ds _ / ge -s{u\x\ 2 coih{u)-itu) 









Now we may apply the identity 

i / ™_i _ „ a . r 



f 00 1 

/ s m -V sA ds = — for Re(,4) > 
Jo A™ y ' 



r(m) 7 

to the above integral, then we have 

r(2) f + °° e- Xu du 



K x (x,t) 



8tt 3 J^oo sinh 2 (n) (\x\ 2 coth(ii) - it) 2 ' 
Denote 

r = (|x| 4 + t 2 )i and = r~ 2 {\x\ 2 - it) 

with E ( — f ) f )• Using the identity 

cosh(ii + i(f>) = cosh(u) cos <j) + i sinh(it) sin (j), 

one has 

1 f°° e~ Xu 
^ ^ a( ) ) 87T 3 y_ o0 [r 2 cosh(n + i0)] 2 

Changing the contour, the formula (|6.4|) becomes 

*aOM) = 5-3-4^ / f £7 M2 d "- 
87r d r 4 j.oo [cosh(u)J 2 
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The above integral can be evaluated as follows: 



K x (x,t) = -^r-^F 



47T 3 
4^3 ] 



V 2 



2 + A 



A 



, 2+A 

it) 2~([a:| +it) 



2-A 

2 



From the above formula, we know that the kernel can be extended from | Re(A)| < 2 to the 
region C \ A where 

A = {±(2 + 2k) : k € Z + } . 

This coincides with the result obtained by Folland and Stein ^0]- In particular, when A = 0, 
one has 

*0(Z, t) = (\X\ 2 + it)-H\x\ 2 - it)-l = JL (| X |4 + ^-l. 

which recovered a result of Folland |Hj ■ 

Remark 6.1. Let us look on Cauchy-Riemann equations on a domain D C H 2 . 
be a smooth function of two quaternion variables, defined in a domain D C I 
Cauchy-Riemann equations is expressed by the system 



Let ii(gi,g 2 ) 
I 2 . Then the 



(6.5) 



B, 



'12 



u 



du 

771- = /1 

771- = 72- 
oq2 



Since the product d qi d q2 is not commutative, we can not obtain the compatibility condition 
on / of the type 

dh = dh 
dqi dq 2 ' 

So the result d 2 7^ is quite different from the Heisenberg group. Moreover, calculations of 
forms for quaternion is more complicated. The simplest anticommutative property 

(6.6) dq\ A dq2 = —dq2 A dq\ 

does not hold. This implies that dq A dq 7^ 0. So even if [H,H] = we can not expect B 2 , 
since (|6.6|) . It was proved in that the system ()6.5|) has a C°° solution u (91,^2) defined in 
a convex domain D C H 2 if and only if the vector (fi, fa) solve a matrix the second order 
differential equation that can be considered as an analogue of the compatibility condition. 
However, can we find an elegant way to describe is the operator □& = dbd^+d^db^ Furthermore, 
what is the correct setting for the (9-Neumann problem on a bounded domain in H 2 ? 



Remark 6.2. How to define correctly the pseudoconvexity in quaternion setting? We need 
the analogue of Hermitian matrix and Levi matrix. Even for the case of 1-dimension it is 
need to understand. We have the tangential Cauchy-Riemann-Fueter operator H, such that 
[H, H] = 2 Ylt=i ikBt k ■ We can say that Yl\=i ^kBt k is an elementary pure imaginary vector 
field defining the missing directions of real dimension 3. Then the Levi matrix is the number 
2, which is positive definite. What is in general case? 
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